MetaLab is a research tool for aggregating across studies in the cognitive development literature. Currently, MetaLab contains 2112 effect sizes across meta-analyses in 25 domains of cognitive development, based on data from 547 papers collecting 38105 subjects. These studies can be used to obtain better estimates of effect sizes across different domains, methods, and ages. Using our power calculator, researchers can use these estimates to plan appropriate sample sizes for prospective studies. More generally, MetaLab can be used as a theoretical tool for exploring patterns in development across language acquisition domains. Learn more here.
Datasets tab.Statistical Approach tab.Field Specification tab.Contribute page.Please note that data and visualizations are under development at the moment (Spring 2018) and should not be taken as definitive.
This page provides an overview of all datasets in MetaLab at this moment. All datasets have more detailed descriptions available through their home pages.
Citation:
N papers = 9, N effect sizes = 80
In a triad-task, bias to generalize to taxonomic as opposed to thematic alternative.
Curator: Molly Lewis
Comments:
Citation: Margoni, F., & Surian, L. (2018). Infants’ evaluation of prosocial and antisocial agents: A meta-analysis. Developmental Psychology, 54(8), 1445-1455.
N papers = 26, N effect sizes = 61
Do infants prefer a prosocial agent (helper, giver, more fair) over an antisocial (hinderer, taker, less fair) agent?
Curator: Francesco Margoni
Comments:
Citation: Christodoulou, J., Lac, A., & Moore, D. S. (2017). Babies and math: A meta-analysis of infants’ simple arithmetic competence. Developmental Psychology, 53(8), 1405-1417.
N papers = 6, N effect sizes = 14
Do infants look longer at stimuli representing “incorrect” versus “correct” solutions of basic addition and subtraction problems?
Curator:
Comments:
Citation: Rabagliati, H., Ferguson, B., & Lew-Williams, C. (2018). The profile of abstract rule learning in infancy: Meta‐analytic and experimental evidence. Developmental Science.
N papers = 20, N effect sizes = 95
Curator: Hugh Rabagliati
Comments:
Cross-situational word learning
Citation: Dal Ben, R., Souza,D. & Hay, J. (2019). Cross-situational word-learning.
N papers = 16, N effect sizes = 50
Infants and children s abilities to learn words across multiple ambiguous situations.
Curator: Rodrigo Dal Ben
Comments:
Citation: Bergmann, C., & Cristia, A. (2016). Development of infants’ segmentation of words from native speech: A meta-analytic approach. Developmental Science, 19, 901–917
N papers = 5, N effect sizes = 17
Recognition of familiarized function words from natural speech using behavioral methods applied to infants 0 - 20 months of age.
Curator: Christina Bergmann
Comments:
Citation: Carbajal, M.J. (2018). Doctoral dissertation
N papers = 16, N effect sizes = 34
Within-subject comparisons of attention to lists of familiar words vs novel/rare words.
Curator: Julia Carbajal
Comments:
Citation: Frank, M. C., Lewis, M., & MacDonald, K. (2016). A performance model for early word-learning. In Proceedings of the 38th Annual Conference of the Cognitive Science Society.
N papers = 12, N effect sizes = 33
Gaze-following using standard multi-alternative forced-choice paradigms, in infants and children from 3 months to two years.
Curator: Molly Lewis
Comments:
Infant directed speech preference
Citation: Dunst, C. J., Gorman, E., & Hamby, D. W. (2012). Preference for infant-directed speech in preverbal young children. Center for Early Literacy Learning, 5(1).
N papers = 23, N effect sizes = 75
Looking times as a function of whether infant-directed vs. adult-directed speech is presented as stimulation, in infants aged 0 - 12 months.
Curator: Alex Cristia
Comments:
Label advantage in concept learning
Citation:
N papers = 18, N effect sizes = 102
Previous research suggests that labels support the formation of novel categories in young children. This meta-analysis includes studies exploring this question that use a paradigm similar to Balaban and Waxman (1997), resulting in a selection of experiments using behavioral methods on infants 3 to 18 months.
Curator: Molly Lewis
Comments:
Citation: Lewis, M., Cristiano, V., Lake, B. M., Kwan, T., & Frank, M. C. (under review). The role of developmental change and linguistic experience in the mutual exclusivity effect.
N papers = 48, N effect sizes = 146
Mutual exclusivity is the bias for children to assume a novel word refers to a novel object, in the presence of one novel and one familiar object. This meta-analysis covers behavioral experiments on infants aged 14 months to 5 years.
Curator: Molly Lewis
Comments:
Citation: Von Holzen, K., & Bergmann, C. (2018). A meta-analysis of infants’ mispronunciation sensitivity development. Proceedings of the 40th Annual Conference of the Cognitive Science Society. Austin, TX: Cognitive Science Society, Inc.
N papers = 32, N effect sizes = 251
Curator: Katie Von Holzen
Comments:
Citation: Issard, C., & Cristia, A. (2019, April 4). Infants prefer to listen to speech: A meta-analysis. Retrieved from osf.io/4stz9.
N papers = 18, N effect sizes = 64
Do infants prefer natural speech over other species vocalizations, natural or artificial sounds?
Curator: Cecile Issard
Comments: This meta-analysis is in progress.
Citation: Frank, M. C., Lewis, M., & MacDonald, K. (2016). A performance model for early word-learning. In Proceedings of the 38th Annual Conference of the Cognitive Science Society.
N papers = 6, N effect sizes = 15
Online word recognition of familiar words using two-alternative forced choice preferential looking. Experiments in this MA follow Fernald et al. (1998) and were included based on their reporting of reaction time as well as effect size.
Curator: Molly Lewis
Comments:
Citation: Cristia, A., (2018). Can infants learn phonology in the lab? A meta-analytic answer. Cognition, 170, 312-327.
N papers = 15, N effect sizes = 47
Laboratory experiments in which infants are exposed to a set of pseudo-words that could support a phonotactic generalization, and thereafter tested on their preference between new types that follow or violate the familiarized phonotactics.
Curator: Alex Cristia
Comments:
Pointing and vocabulary (longitudinal)
Citation: Colonnesi, C., Stamsa, G. J., Kostera, I., & Noomb, M. J. (2010). The relation between pointing and language development: A meta-analysis. Developmental Review, 30, 352–366.
N papers = 18, N effect sizes = 18
The pointing gesture has been argued to be related to the development of language. This meta-analysis explores the relationship between pointing and language development, both concurrently and longitudinally.
Curator: Molly Lewis
Comments:
Pointing and vocabulary (concurrent)
Citation: Colonnesi, C., Stamsa, G. J., Kostera, I., & Noomb, M. J. (2010). The relation between pointing and language development: A meta-analysis. Developmental Review, 30, 352–366.
N papers = 12, N effect sizes = 12
The pointing gesture has been argued to be related to the development of language. This meta-analysis explores the relationship between pointing and language development, both concurrently and longitudinally.
Curator: Molly Lewis
Comments:
Statistical sound category learning
Citation: Cristia, A., (2018). Can infants learn phonology in the lab? A meta-analytic answer. Cognition, 170, 312-327.
N papers = 11, N effect sizes = 20
Laboratory experiments in which infants are exposed to a set of syllables that have been generated using a bimodal distribution of acoustic cues or not, and infants are tested on their discrimination of two sounds differing along the same dimension.
Curator: Alex Cristia
Comments:
Citation: Black, A. & Bergmann, C. (2017). Quantifying infants’ statistical word segmentation: A meta-analysis. In: Proceedings of the 39th Annual Conference of the Cognitive Science Society, pp. 124-129, Austin, TX: Cognitive Science Society
N papers = 31, N effect sizes = 103
Can infants segment words from artificial mini-languages based on syllable-level statistics?
Curator: Christina Bergmann
Comments:
Citation: Fort, M., Lammertink, I., Peperkamp, S., Guevara-Rukoz, A., Fikkert, P., & Tsuji, S. (2018). SymBouki: a meta-analysis on the emergence of sound symbolism in early language acquisition. Developmental Science, Early View. doi: 10.1111/desc.12659.
N papers = 11, N effect sizes = 44
Do infants recognize systematic relationships between form and meaning, e.g. that bouba tends to refer to a round object and kiki tends to refer to a spiky one?
Curator: Sho Tsuji
Comments:
Citation: Tsui, A. S. M., Byers-Heinlein, K., & Fennell, C. T. (2019, February 7). Associative Word Learning in Infancy: A Meta-Analysis of the Switch Task. Developmental Psychology. Advance online publication.
N papers = 47, N effect sizes = 143
Curator: Angeline Tsui
Comments:
Citation: Quinn, S., Donnelly, S., & Kidd, E. (2018). The relationship between symbolic play and language acquisition: A meta-analytic review. Developmental review, 49, 121-135.
N papers = 31, N effect sizes = 196
Do infants who show more symbolic play have better language development?
Curator:
Comments: the mean age is not systematic, but the center of the age range provided in the supplementary material.
Citation: Tsuji, S. & Cristia, A. (2014). Perceptual attunement in vowels: A meta-analysis. Developmental Psychobiology, 56(2), 179-191.
N papers = 33, N effect sizes = 145
Discrimination of native-language vowels, including results from behavioral, electrophysiological, and neuroimaging methods applied to infants 0 - 14 months of age.
Curator: Sho Tsuji
Comments:
Vowel discrimination (non-native)
Citation: Tsuji, S. & Cristia, A. (2014). Perceptual attunement in vowels: A meta-analysis. Developmental Psychobiology, 56(2), 179-191.
N papers = 15, N effect sizes = 49
Discrimination of non-native language vowels, including results from behavioral, electrophysiological, and neuroimaging methods applied to infants 0 - 14 months of age.
Curator: Sho Tsuji
Comments:
Citation: Bergmann, C., & Cristia, A. (2016). Development of infants’ segmentation of words from native speech: A meta-analytic approach. Developmental Science, 19, 901–917
N papers = 68, N effect sizes = 298
Recognition of familiarized content or non-words from running, natural speech using behavioral methods applied to infants 0 - 20 months of age.
Curator: Christina Bergmann
Comments:
We welcome researchers interested in contributing to Metalab. Please contact us.
Contributions can take various forms:
Check out the tutorial page on how to edit your meta-analysis in the MetaLab format.
All analyses on the site are conducted with the metafor package (Viechtbauer, 2010).
Effect size computation is handled by a script, compute_es.R.
Several pre-existing MAs deal with special cases, and these are listed in the script.
Except where noted, formulas are from Hedges & Olkin’s textbook.
The visualizations page uses a multi-level random effects meta-analysis ( rma.mv function of metafor). Random-effect models assume that the true effect can vary between different studies, and therefore allow random effects for each data point. The model also specifies random effects on the level of each paper, since studies within a paper can be assumed to be more similar to each other than to studies from different papers. In addition, we allow correlated random effects within each paper to account for cases where the same infants contributed data to multiple rows.
The meta-analytic models are accessible in the script server.R
MetaLab promotes open science practices. All the datasets of the meta-analyses appearing on the website can be downloaded. If you want to work with one of them, these instructions might be useful:
This page gives the full specification for each field in the metalab dataset, including: required fields (which must be included for every MA), optional fields (which are only used for some MAs), and derived fields (which are computed by the site).
You can download the data from both the home page and the visualization page.
From the home page
1.1. Click on the box corresponding to the domain you are interested in (e.g. cognitive development, early language).
1.2. Once you are on the page of the domain, click on the box corresponding to the meta-analysis you are interested in (e.g. word segmentation).
1.3. In the data tab, click on the “Download” button.
1.4. Choose the data format that you want (EXCEL - recommended for manual calculations using a spreadsheet software - but you should make sure that your local spreadsheet software is set to use “.” as decimal separator to avoid errors (see section 2.), or CSV - recommended for reading into a statistical software such as R) by clicking on it in the conditional menu.
From the visualization page.
2.1. Open the “Domain” drop-down menu and click on the one you are interested in.
2.2. Open the “Dataset” drop-down menu and click on the meta-analysis you are interested in.
2.3. Click on the “Download data” button above the “Domain” menu. You can also click on “View raw dataset” and follow steps 1.2. and 1.3.
The following instructions assume you are using Excel:
Open a blank workbook
Go to the data menu
Click on “Get external data”
Click on “From text”
A dialog box opens. In the drop-down menu on the right, select “UTF-8”.
Select “Delimited” and click “next”
Select “comma” as column delimiter and click “next”
Click on the the “advanced” button and check that “.” is the decimal separator.
Click OK to finish.
Your research question may be more specific than the one of the meta-analysis. For example your question may focus on a more specific age than the meta-analysis, or you are using a specific method.
Click on “Data” > “Filter” (libre/openoffice have similar menus)
Scroll to the column coding your criteria (e.g. “mean_age_1”)
Click on the little triangle/filter symbol on the right corner of that column
Click on “Standard filter” or “Special filter” or “Custom filter”
In the next dialogue box, choose your condition to be that the column entitled “mean_age_1” is lower than the maximum age (in days) your participants-to-be will have, and greater than your lower age bound.* (We use the conversion: month x 30.42 days/month). If you want to add a second condition, don’t press OK just yet!
To add a second condition: that the “method” is your chosen method (see here for codes). You SHOULD keep only studies with your chosen method, but other variables are left to your judgement. For instance, if I were doing a study on 6 mo using Central Fixation, then my filters could be: method=“CF” AND 120 < mean_age_1 <=365 (so babies between about 4 months and 1 year).
You can add as many conditions as you want (e.g., exclude some more studies because they use unusual stimuli, or a very different design from yours).
When you’re done entering inclusion conditions, you can click OK. The result will be a set of rows that contain all the effect sizes meeting your conditions.
Two possibilities:
Compute the mean or the median of column “n_1”, and “n_1” and “n_2” if you are running a between participants study. You can thus see how many infants per group you should test based on the sample sizes used in previous studies. We do not recommend this strategy.
1.1. In a new cell (typically below the last row of the “n_1” and “n_2” columns), type “=Mean([coordinate (i.e. letter+number) of the first value]:[coordinate (i.e. letter+number) of the last value])” or “=median([coordinate (i.e. letter+number) of the first value]:[coordinate (i.e. letter+number) of the last value])”. You can also type “=mean(“ and select the the cells to average with the cursor. Make sure you’ve closed the bracket and press enter.
1.2. Note that if the variable “n_excluded_1” is coded in the meta-analysis you are looking at, you can also get an idea of the attrition rates found in this work. For instance, you can add a column, say called “n_total”, that sums n_1 and n_excluded_1. This will give you an idea of how many babies you should recruit, taking into account that some of them will need to be excluded (because they are fussy or other reasons).
1.3. If n_excluded_1 is not present, you can use a 20% rule of thumb - i.e., recruit 20% more than your target sample size. You can get an idea of the attrition rate by method and age group on our design choices analyses page.
Previous studies might have chosen their sample size based on practical, instead of statistical reasons -e.g. number of available participants within a predefined time range. Therefore they might be underpowered. To solve the problem on your study, you can do a prospective power analysis.
2.1. Copy the column entitled d_calc into a new sheet.
2.2. Calculate the median values for these columns – in my example, the median difference score is 0.91, the median pooled SD is 2.72, and the median effect size is 0.33.
2.3. Use your favorite power calculation system/online tool to estimate sample size. I used the following formula in R (package: pwr): power.t.test(delta=0.91,sd=2.72,power=0.8,sig=.05,type=c(“paired”),alternative=“two.sided”) to find out I needed to test 72 infants for my example case.
If the number you get is too high, you might consider running sequential analyses. This means that you fully design your study, including a predetermined number of participants to run, the statistical analyses that you will run, and predefine moments when you will look at p-values before the end of data collection. At these predefined moments you will decide to continue or stop data collection based on outcome (significance reached, or fell below the threshold).
You can also define a stopping rule related to your data collection situation - e.g. “Data collection will stop when all conditions are balanced (N=72) or by Dec 31, 2018 (date when my internship ends), whichever happens first”. Notice that neither of these options takes power into consideration - they are not alternatives to power, they just help you avoid the questionable practice of stopping data collection when you have p<.05.
You must cite the publications linked to that dataset, as listed in the Documentation. (If there is no citation, then no citation for the individual dataset is necessary).
For tracking use of MetaLab, please also cite at least one of the following:
Bergmann, C., Tsuji, S., Piccinini, P.E., Lewis, M.L., Braginsky, M., Frank, M.C., & Cristia, A. (2018). Promoting replicability in developmental research through meta-analyses: Insights from language acquisition research. Child Development, 89, 1996-2009 . DOI: 10.1111/cdev.13079(https://osf.io/uhv3d/)[Repository]
Lewis, M. L., Braginsky, M., Tsuji, S., Bergmann, C., Piccinini, P. E., Cristia, A., & Frank, M. C. (2017/under review). A Quantitative Synthesis of Early Language Acquisition Using Meta-Analysis. DOI: 10.17605/OSF.IO/HTSJM (https://psyarxiv.com/htsjm)[Preprint]
If researchers use more than five datasets, we ask users as a courtesy to cite all data. But in case of severe space limitations, the database as a whole alone may be cited.
Since the MetaLab site is dynamic, we recommend that you list the date of download for your data in your manuscript. For example: “We analyze all data currently in MetaLab (Bergmann et al., 2018). Data were downloaded on 06/17/17.”
We are collecting frequent problems here. If you experience any issues that we do not cover when following the steps, please send an email to [metalab-project@googlegroups.com] and we will try to help you.
Make sure you selected UTF-8 as the encoding when you imported the data (see steps 2.1 to 2.5).
This is due to a faulty import, you need to do it again:
Open a blank workbook
Go to the data menu
Click on “Get external data”
Click on “From text”
A dialog box opens. In the drop-down menu on the right, select “UTF-8”.
Select “Delimited” and click “next”
Select “comma” as column delimiter and click “next”
Click on the the “advanced” button and check that “.” is the decimal separator.
Click OK to finish.
Check that . is the decimal separator:
Select the column where numbers are not formatted properly (if there are several, you need to treat them one by one).
Click on “data” > “Text to columns”. A dialog box opens
Select “Delimited” and click “next”
Select “comma” as column delimiter and click “next”
Click on the the “advanced” button and check that “.” is the decimal separator. Click OK to finish.
select column A (that contains all the data)
Click on “data” > “Text to columns”. A dialog box opens
Select “Delimited” and click “next”
Select “comma” as column delimiter and click “next”
Click on the the “advanced” button and check that “.” is the decimal separator. Click OK to finish.